1. Motivation 

2. Mask Finding and Scaling 

3. Richardson-Lucy Deconvolution 
4. Recovering Intensity 

5. Results and Remarks 


Motivation 


Optical blurring of an image is the result of light rays from a single object 
focusing at a point before or after the sensor in a camera. This physical 
situation can be described as a convolution of the true captured image with 
an optical filter: the point spread function (PSF). 

Equation: 


Given that the PSF works much like a frequency response for a spatial 
domain, we explore the possibility of extracting a focused image from an 
image that has been captured out of focus. 

Equation: 
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where D, K, I are the transforms of the deblurred image, the PSF, and a 
blurred image. 


Instead of using the full circular aperture of most cameras, we place a coded 
mask within the lens of our camera. The mask itself is nothing more than a 
binary-coded transparency, but it enables us to force a particular PSF into 
our camera system. By recreating the mask's code in software, we can 
recover the PSF that is appropriate for the depth at which our image would 
have been focused. Once we have the appropriate PSF for a particular 
focus-depth (which correlates to increasing or decreasing the size of the 
mask in software), we use the Richardson-Lucy Deconvolution algorithm to 
recover a focused image. 


Mask Finding and Scaling 


The PSF is effectively the impulse response in the optical domain. If we try 
to describe a system with an incorrect impulse response, the output is not 
true. Accordingly, every plane at a particular depth has its own impulse 
response, or PSF in this case. The size or scale of a PSF corresponds to a 
particular plane of focus in space and a blurred image is effectively 
captured when a lens is focused at a depth that does not correspond to the 
plane in which the object exists. This is the same situation we recreate when 
we capture our blurred images. 


For these reasons it is important that we obtain not only an exact replica of 
the PSF used to capture the blurred image, but also the exact scale for the 
depth of the captured image. An incorrect PSF yields less than desirable 
results and after experimenting with various techniques we developed two 
methods for generating the PSF for a given mask and depth in MATLAB: 
manual construction of the matrix and a search algorithm. 
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Original Matrix of Scale 1 is transformed to a Matrix of Scale 2. 


When the distance from the camera to the object is unknown, manual 
construction of a mask is preferred. The small size of this method allows 


the mask to be scaled easily as shown above so that the appropriate PSF for 
a given distance can be searched. The masks are represented by an N-by-N 
matrix, where every black block is marked as a 0 and every transparent 
block is marked as a 1. 


The simplicity of this method is appealing and provides decent results. One 
of the main drawbacks to creating the matrix manually is that the actual 
mask hardly ever appears exactly in the image. For example, the corners of 
our masks are rounded by the camera aperture. We are able to compensate 
for this by reducing the intensity of the comer entries in our matrix to about 
0.5. A second drawback is that rotations in the placement of the mask in the 
camera reduce the effectiveness of our manually constructed matrix. 


Auto Mask 


Captured PSF (Left) and resulting mask (right) using Automask. 


After capturing an image of the appropriate PSF for the depth at which our 
object exists, we isolate the PSF in MATLAB. First the PSF image is 


converted to black and white. Given the simplicity of our algorithm, it is 
necessary to apply some smoothing to this black and white image. After, it 
is cropped and reproduced automatically using a 0-1 search algorithm, 
yielding a high-resolution matrix of the appropriate PSF. This method 
eliminates the need for scaling and much of the rigor required to create the 
PSF manually, such as aligning the mask exactly. Overall the mask that is 
recovered by this method is truer to the actual mask and results in better 
deblurring. 


Then What? 

Once a PSF is obtained, either by Hand Mask or Auto Mask, we can deblur 
using the blurred image, the PSF, and the Richardson-Lucy Deconvolution 
Algorithm. 


Richardson-Lucy Deconvolution 


We used the Richardson-Lucy algorithm to recover our true image that has 
been blurred by our point spread function. Our observed image di is the 
sum of our point spread function, which is the fraction of light observed at 
position i originating from true location j) multiplied by the pixel value uj 
of the true image. Photon noise is Poisson distributed, therefore the 
Richardson-Lucy algorithm assumes that all uj pixel values are Poisson 
distributed. 


We can iteratively solve for the most likely uj according to the formula: 
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Because our masks are complicated, Richardson-Lucy will create spatial 
ringing since we have multiple locations in which light can be transmitted 
through and it does not always correctly determine the origin of that 
particular ray. This is why the pinhole aperture deblurred image contains no 
ringing artifacts because the Richardson-Lucy algorithm will correctly find 
the true image as the single point because that is always the maximum. 


Recovering Intensity 


Image captured using a pinhole mask (left) and the deblurred image 
(right) without light correction. 


The mask reduces the amount of light that enters the camera aperture, 
reducing the brightness of the captured images. The deblurred image we 
recover is dim compared to the same image captured without the coded 
aperture so we compensate by scaling the intensity of the deblurred image 
according to the ratio of opaque area in the mask to the total area of the 
mask. The intensity of the deconvolved image is extracted, scaled, then 
replaced for a more appealing image. 


Results and Remarks 
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Visually our deblurred 
images appear to have 
good results, but SNA 
measurements with a 
clear natural image 


or 
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Conclusions and Suggestions 

As can be seen from the images above, our efforts produce qualitative and 
quantitative results. Clearly the image of the bear shows significant 
deblurring and the SNR supports that our methods have chosen the 
appropriate PSF to do so. 


The SNR table shows that there are small differences between scales. This 
is aresult of some downsampling we used to simplify the analysis process 
and should not suggest that any arbitrary scale may be used to deblur. In 
reality, there is a meaningful difference in the SNRs of various scales. 


Finally, we would like to suggest an extension to our project. We were only 
able to successfully deblur the images of objects that existed in a flat plane, 
at one depth, in other words. The applications of this method could be 
extended significantly by developing a way to characterize the different 
depths of a real life image. A depth-map would provide the means to use 
the correct PSF for every point in the image, producing an all-focus image. 


